Performing Exploratory Data Analysis and Generating Analytical Reports in Python
Course: Python Training
Topic: Performing Exploratory Data Analysis and Generating Analytical Reports
Course Link: Python Training Course
Demo Link: Register for a Course Demo
Website: JustAcademy
1. Introduction to Exploratory Data Analysis
Exploratory Data Analysis (EDA) is the process of examining, summarizing, and visualizing data to understand its structure, discover patterns, identify relationships, and detect potential problems before conducting deeper statistical analysis or building machine learning models.
EDA helps answer important questions about a dataset: What information does it contain? Are there missing or duplicated records? Which variables have the highest values? Are there trends over time? Do certain categories perform better than others?
Python is widely used for EDA because libraries such as pandas, NumPy, Matplotlib, and Seaborn provide convenient tools for data inspection, calculations, statistical summaries, and visualization.
Objectives of EDA
- Understand the size, structure, and data types of a dataset.
- Identify missing values, duplicate records, and inconsistent entries.
- Calculate descriptive statistics for numerical and categorical variables.
- Discover patterns, trends, relationships, and unusual observations.
- Compare performance across categories, regions, or time periods.
- Prepare reliable information for analytical reports and business decisions.
2. Understanding Analytical Reports
An analytical report is a structured document that presents data findings, important metrics, visualizations, interpretations, and recommendations. Its purpose is to turn raw data into information that readers can understand and use.
For example, a retail sales report may explain total revenue, monthly sales trends, best-performing products, regional performance, and opportunities for improvement.
Components of an Analytical Report
| Component |
Description |
| Report title |
Identifies the topic and scope of the analysis. |
| Objective |
Explains the question or business problem being investigated. |
| Dataset overview |
Describes the data source, time period, rows, and columns. |
| Data-quality assessment |
Documents missing values, duplicates, and data limitations. |
| Key performance indicators |
Summarizes important business measurements. |
| Visualizations |
Displays trends, distributions, comparisons, and relationships. |
| Findings |
Explains what the data reveals. |
| Recommendations |
Suggests actions supported by the findings. |
| Limitations |
Identifies assumptions, incomplete information, and constraints. |
3. Python Libraries Used for EDA
3.1 pandas
Pandas is used to load, filter, clean, group, transform, and summarize structured data using Series and DataFrames.
import pandas as pd
df = pd.read_csv("sales.csv")
print(df.head())
3.2 NumPy
NumPy provides numerical arrays and mathematical functions for statistical calculations and numerical analysis.
import numpy as np
values = np.array([100, 200, 300, 400])
print("Mean:", np.mean(values))
print("Median:", np.median(values))
print("Standard deviation:", np.std(values))
3.3 Matplotlib
Matplotlib is used to create line charts, bar charts, histograms, scatter plots, and other visualizations.
import matplotlib.pyplot as plt
plt.plot([1, 2, 3, 4], [10, 15, 12, 20])
plt.title("Sample Trend")
plt.xlabel("Period")
plt.ylabel("Value")
plt.tight_layout()
plt.show()
3.4 Seaborn
Seaborn simplifies statistical visualization and supports charts such as box plots, heatmaps, histograms, and scatter plots.
import seaborn as sns
import matplotlib.pyplot as plt
sns.histplot(data=df, x="Sales", kde=True)
plt.title("Sales Distribution")
plt.tight_layout()
plt.show()
3.5 Installing Required Libraries
pip install pandas numpy matplotlib seaborn openpyxl
Run the command in your terminal or Python environment. The openpyxl package supports many Excel file operations.
4. Loading Data into Python
Data can come from CSV files, Excel workbooks, databases, APIs, and other structured sources. The loading method depends on the file format and data source.
4.1 Read a CSV File
import pandas as pd
df = pd.read_csv("sales.csv")
print(df.head())
4.2 Read an Excel File
df = pd.read_excel("sales.xlsx")
print(df.head())
4.3 Read a Specific Excel Sheet
df = pd.read_excel(
"sales.xlsx",
sheet_name="Monthly Sales"
)
4.4 Load Data from SQLite
import sqlite3
import pandas as pd
connection = sqlite3.connect("business.db")
try:
query = "SELECT * FROM sales"
df = pd.read_sql_query(query, connection)
finally:
connection.close()
print(df.head())
When loading data, verify the file path, encoding, column names, and date formats. For large datasets, consider loading only the required columns or processing the data in chunks.
5. Inspecting Dataset Structure
Before calculating metrics, inspect the dataset to understand its dimensions, columns, data types, and sample records.
5.1 Display the First and Last Rows
print(df.head())
print(df.tail())
head() displays the first five rows by default, while tail() displays the last five rows.
5.2 Check the Number of Rows and Columns
print("Dataset shape:", df.shape)
print("Rows:", df.shape[0])
print("Columns:", df.shape[1])
5.3 Inspect Column Names
print(df.columns.tolist())
Column names should be clear and consistent. Remove unwanted whitespace when necessary:
df.columns = df.columns.str.strip()
5.4 Inspect Data Types and Non-Missing Values
df.info()
This method displays column names, data types, non-missing counts, and memory usage. It helps identify columns that may have been imported using incorrect data types.
5.5 Check Unique Values
for column in df.columns:
print(f"\nColumn: {column}")
print("Unique values:", df[column].nunique(dropna=True))
Unique-value counts help identify categorical variables, potential identifiers, and columns with very few distinct values.
6. Descriptive Statistics
Descriptive statistics summarize the central tendency, spread, and range of values in a dataset.
6.1 Generate Summary Statistics
print(df.describe())
For numeric columns, describe() typically returns the count, mean, standard deviation, minimum, quartiles, and maximum.
6.2 Include Categorical Columns
print(df.describe(include="all"))
For categorical columns, the summary may include the number of unique values, the most frequent value, and its frequency.
6.3 Calculate Individual Statistics
print("Mean:", df["Sales"].mean())
print("Median:", df["Sales"].median())
print("Minimum:", df["Sales"].min())
print("Maximum:", df["Sales"].max())
print("Standard deviation:", df["Sales"].std())
6.4 Understanding Common Statistics
| Statistic |
Meaning |
Application |
| Mean |
Arithmetic average. |
Average sales per transaction. |
| Median |
Middle value in ordered data. |
Typical order value when outliers exist. |
| Mode |
Most frequently occurring value. |
Most popular product category. |
| Minimum |
Smallest observed value. |
Lowest transaction amount. |
| Maximum |
Largest observed value. |
Highest transaction amount. |
| Standard deviation |
Measures variability around the mean. |
Variation in sales values. |
| Quartiles |
Divide ordered observations into four sections. |
Understanding data distribution. |
7. Assessing Data Quality
Data-quality checks are important because missing values, duplicate records, and incorrect entries can distort statistics and visualizations.
7.1 Count Missing Values
missing_counts = df.isna().sum()
print(missing_counts)
7.2 Calculate Missing-Value Percentages
missing_percentages = (
df.isna().mean() * 100
).round(2)
print(missing_percentages)
7.3 Detect Duplicate Rows
print("Duplicate rows:", df.duplicated().sum())
7.4 Review Categorical Values
print(df["Category"].value_counts(dropna=False))
print(df["Category"].unique())
7.5 Check Data Types
print(df.dtypes)
Use the results to determine whether values need to be standardized, converted, filled, or investigated. Do not automatically remove every unusual record because some unusual values may be legitimate.
8. Univariate Analysis
Univariate analysis examines one variable at a time. It helps identify distributions, common values, variation, and potential outliers.
8.1 Analyze a Numeric Variable
print(df["Sales"].describe())
print("Median sales:", df["Sales"].median())
print("Unique sales values:", df["Sales"].nunique())
8.2 Create a Histogram
import matplotlib.pyplot as plt
df["Sales"].dropna().plot(
kind="hist",
bins=20,
edgecolor="black"
)
plt.title("Distribution of Sales")
plt.xlabel("Sales Amount")
plt.ylabel("Frequency")
plt.tight_layout()
plt.show()
A histogram groups numeric values into intervals. It helps identify skewness, clusters, multiple peaks, and potential outliers.
8.3 Analyze a Categorical Variable
category_counts = df["Category"].value_counts()
print(category_counts)
8.4 Create a Bar Chart
import matplotlib.pyplot as plt
category_counts = df["Category"].value_counts()
category_counts.plot(kind="bar")
plt.title("Records by Category")
plt.xlabel("Category")
plt.ylabel("Number of Records")
plt.xticks(rotation=45)
plt.tight_layout()
plt.show()
Bar charts help compare counts or summary values across categories.
9. Bivariate Analysis
Bivariate analysis examines two variables to understand their relationship or compare their behavior.
9.1 Calculate Correlation
correlation = df[
["AdvertisingSpend", "Sales"]
].corr()
print(correlation)
Correlation measures the strength and direction of a linear association between two numeric variables. A coefficient near 1 indicates a strong positive linear relationship, while a coefficient near -1 indicates a strong negative linear relationship. A value near zero indicates little linear association.
Correlation does not prove causation. Other variables, unusual observations, or nonlinear relationships may influence the result.
9.2 Create a Scatter Plot
import seaborn as sns
import matplotlib.pyplot as plt
sns.scatterplot(
data=df,
x="AdvertisingSpend",
y="Sales"
)
plt.title("Advertising Spend vs Sales")
plt.tight_layout()
plt.show()
A scatter plot helps reveal clusters, nonlinear patterns, and unusual observations.
9.3 Compare a Numeric Variable Across Categories
category_sales = df.groupby("Category")["Sales"].mean()
print(category_sales)
This calculates average sales for each category. If the data is skewed or category sizes differ substantially, also examine medians, sample sizes, and distributions.
10. Multivariate Analysis
Multivariate analysis examines several variables together. It can reveal patterns that are not visible when analyzing individual columns separately.
10.1 Create a Correlation Matrix
numeric_df = df.select_dtypes(include="number")
correlation_matrix = numeric_df.corr()
print(correlation_matrix)
10.2 Visualize Correlations with a Heatmap
import seaborn as sns
import matplotlib.pyplot as plt
sns.heatmap(
correlation_matrix,
annot=True,
cmap="coolwarm",
fmt=".2f"
)
plt.title("Correlation Matrix")
plt.tight_layout()
plt.show()
A heatmap makes it easier to compare pairwise correlations among numeric variables. Interpret these results alongside sample sizes, data quality, and domain knowledge.
10.3 Compare Multiple Groups
summary = df.groupby(
["Region", "Category"]
).agg(
Total_Sales=("Sales", "sum"),
Average_Sales=("Sales", "mean"),
Transaction_Count=("Sales", "count")
)
print(summary)
This example compares categories within regions and calculates several metrics in one operation.
11. Time-Series Exploratory Data Analysis
Time-series analysis examines observations recorded over time, such as daily revenue, monthly sales, or hourly website traffic. It helps identify trends, seasonal patterns, fluctuations, and unusual events.
11.1 Convert Dates to Datetime
df["OrderDate"] = pd.to_datetime(
df["OrderDate"],
errors="coerce"
)
df = df.sort_values("OrderDate")
11.2 Group Sales by Month
monthly_sales = (
df.set_index("OrderDate")["Sales"]
.resample("ME")
.sum()
)
print(monthly_sales)
The ME frequency groups values by calendar month-end in current pandas versions. Confirm that dates and sales values are valid before interpreting totals.
11.3 Visualize Monthly Sales
import matplotlib.pyplot as plt
monthly_sales.plot(kind="line", marker="o")
plt.title("Monthly Sales Trend")
plt.xlabel("Month")
plt.ylabel("Total Sales")
plt.tight_layout()
plt.show()
A line chart helps identify upward or downward trends. Seasonal patterns should be investigated using sufficient historical data rather than inferred from only a few observations.
12. Grouping Data and Calculating Business Metrics
The pandas groupby() method divides records into groups and calculates statistics for each group. It is useful for comparing regions, products, departments, customer segments, and sales channels.
12.1 Calculate Total Sales by Region
regional_sales = df.groupby("Region")["Sales"].sum()
print(regional_sales)
12.2 Calculate Multiple Metrics
summary = df.groupby("Region").agg(
Total_Sales=("Sales", "sum"),
Average_Sales=("Sales", "mean"),
Minimum_Sales=("Sales", "min"),
Maximum_Sales=("Sales", "max"),
Transaction_Count=("Sales", "count")
)
print(summary)
The transaction count in this example counts non-missing sales values. It may differ from the number of distinct orders if one order contains multiple product lines.
12.3 Identify Best-Selling Products
product_sales = (
df.groupby("Product")["Sales"]
.sum()
.sort_values(ascending=False)
)
print(product_sales.head(10))
This identifies the ten products with the highest total sales. Revenue alone does not measure profitability, so include cost or profit metrics when evaluating overall performance.
13. Calculating Key Performance Indicators
Key Performance Indicators (KPIs) are measurable values used to evaluate progress toward a business objective. The correct KPI depends on the business model and the question being investigated.
| KPI |
Meaning |
Calculation |
| Total Revenue |
Total revenue generated. |
Sum of revenue values. |
| Average Order Value |
Average revenue per order. |
Total revenue divided by valid order count. |
| Order Count |
Number of distinct orders. |
Count of unique order IDs. |
| Profit |
Revenue minus applicable costs. |
Total revenue minus total cost. |
| Profit Margin |
Profit as a percentage of revenue. |
Profit / Revenue × 100. |
| Growth Rate |
Relative change between two periods. |
(Current - Previous) / Previous × 100. |
13.1 Calculate Total Revenue
total_revenue = df["Sales"].sum()
print(f"Total Revenue: {total_revenue:,.2f}")
13.2 Calculate Average Order Value
# Valid when each row represents exactly one order
average_order_value = df["Sales"].mean()
print(f"Average Order Value: {average_order_value:,.2f}")
If one order contains multiple product rows, first aggregate sales by order ID and then calculate the average of order totals. Otherwise, the calculation may represent average line-item sales rather than average order value.
13.3 Calculate Profit Margin
total_revenue = df["Revenue"].sum()
total_profit = df["Profit"].sum()
if total_revenue != 0:
profit_margin = total_profit / total_revenue * 100
print(f"Profit Margin: {profit_margin:.2f}%")
else:
print("Profit margin cannot be calculated because revenue is zero.")
13.4 Calculate Growth Rate
previous_period = 100000
current_period = 125000
if previous_period != 0:
growth_rate = (
(current_period - previous_period)
/ previous_period
) * 100
print(f"Growth Rate: {growth_rate:.2f}%")
The example shows 25% growth from 100,000 to 125,000. A zero or negative baseline requires careful interpretation according to the metric.
14. Detecting Potential Outliers
An outlier is an observation that differs substantially from most other observations. It may indicate a data-entry error, an unusual but valid event, or an important business occurrence.
14.1 Create a Box Plot
import seaborn as sns
import matplotlib.pyplot as plt
sns.boxplot(data=df, x="Sales")
plt.title("Sales Outlier Inspection")
plt.tight_layout()
plt.show()
14.2 Use the Interquartile Range Method
The interquartile range (IQR) is the difference between the third quartile (Q3) and first quartile (Q1). A common exploratory rule flags values below Q1 − 1.5 × IQR or above Q3 + 1.5 × IQR.
q1 = df["Sales"].quantile(0.25)
q3 = df["Sales"].quantile(0.75)
iqr = q3 - q1
lower_bound = q1 - 1.5 * iqr
upper_bound = q3 + 1.5 * iqr
outliers = df[
(df["Sales"] < lower_bound)
| (df["Sales"] > upper_bound)
]
print(outliers)
This method flags potential outliers; it does not prove that the records are incorrect. Investigate them before deciding whether to retain, transform, or exclude them.
15. Creating Effective Visualizations
Visualizations communicate analytical findings clearly. Each chart should answer a specific question and include a descriptive title, readable labels, appropriate units, and a suitable scale.
| Chart Type |
Purpose |
Example |
| Bar chart |
Compare categories. |
Revenue by region. |
| Line chart |
Show changes over time. |
Monthly sales trends. |
| Histogram |
Explore numeric distributions. |
Order value distribution. |
| Box plot |
Compare distributions and identify potential outliers. |
Sales across categories. |
| Scatter plot |
Explore relationships between numeric variables. |
Advertising spending versus sales. |
| Heatmap |
Display a matrix of values. |
Correlation matrix. |
| Stacked bar chart |
Compare totals and category composition. |
Sales channels by region. |
15.1 Create a Horizontal Bar Chart
import matplotlib.pyplot as plt
region_sales = df.groupby("Region")["Sales"].sum()
region_sales.sort_values().plot(kind="barh")
plt.title("Total Sales by Region")
plt.xlabel("Total Sales")
plt.ylabel("Region")
plt.tight_layout()
plt.show()
A good chart avoids unnecessary decoration and makes comparisons easy. If many categories exist, consider sorting the bars or displaying only the most relevant categories.
16. Building an Analytical Summary Table
A summary table brings important metrics together in one place. It can be displayed in a notebook or exported to CSV, Excel, or HTML.
summary_table = pd.DataFrame({
"Metric": [
"Total Sales",
"Average Sales",
"Median Sales",
"Maximum Sales",
"Minimum Sales",
"Missing Sales Values"
],
"Value": [
df["Sales"].sum(),
df["Sales"].mean(),
df["Sales"].median(),
df["Sales"].max(),
df["Sales"].min(),
df["Sales"].isna().sum()
]
})
print(summary_table)
This example assumes that the dataset contains a numeric column named Sales. Adapt the metrics and column names to the dataset being analyzed.
17. Exporting Analytical Reports
Python can export analytical results into several formats. CSV is useful for tabular data, Excel supports multiple worksheets, HTML is suitable for browser-based reports, and image files can store charts for presentations.
17.1 Export a Summary to CSV
summary_table.to_csv(
"analytical_summary.csv",
index=False
)
17.2 Export Results to Excel
with pd.ExcelWriter("analytical_report.xlsx") as writer:
summary_table.to_excel(
writer,
sheet_name="Summary",
index=False
)
regional_sales.rename("Total Sales").to_frame().to_excel(
writer,
sheet_name="Regional Sales"
)
The workbook contains separate worksheets for the overall summary and regional sales. Install a compatible Excel-writing engine, such as openpyxl, if needed.
17.3 Export a Table as HTML
html_table = summary_table.to_html(
index=False,
border=1
)
with open(
"analytical_summary.html",
"w",
encoding="utf-8"
) as file:
file.write("<h1>Analytical Summary</h1>\\n")
file.write(html_table)
The generated file can be opened in a web browser. A complete report should also include the objective, data-quality results, charts, interpretations, and recommendations.
17.4 Save a Chart as an Image
import matplotlib.pyplot as plt
region_sales.plot(kind="bar")
plt.title("Total Sales by Region")
plt.xlabel("Region")
plt.ylabel("Sales")
plt.tight_layout()
plt.savefig(
"regional_sales.png",
dpi=150,
bbox_inches="tight"
)
plt.close()
Save the figure before closing it. Select an appropriate resolution for the report or presentation.
18. Creating a Reusable EDA Report
A reusable function can automate basic data-quality checks and descriptive statistics for different datasets.
import pandas as pd
def generate_eda_report(df, output_prefix="eda_report"):
if df.empty:
raise ValueError("The dataset contains no rows.")
overview = pd.DataFrame({
"Metric": [
"Number of Rows",
"Number of Columns",
"Duplicate Rows"
],
"Value": [
len(df),
len(df.columns),
int(df.duplicated().sum())
]
})
quality = pd.DataFrame({
"Data Type": df.dtypes.astype(str),
"Missing Count": df.isna().sum(),
"Missing Percentage": (
df.isna().mean() * 100
).round(2),
"Unique Values": df.nunique(dropna=True)
})
statistics = df.describe(
include="all"
).transpose()
overview.to_csv(
f"{output_prefix}_overview.csv",
index=False
)
quality.to_csv(
f"{output_prefix}_quality.csv"
)
statistics.to_csv(
f"{output_prefix}_statistics.csv"
)
html = f"""
<html>
<head>
<meta charset="utf-8">
<title>Exploratory Data Analysis Report</title>
</head>
<body>
<h1>Exploratory Data Analysis Report</h1>
<h2>Dataset Overview</h2>
{overview.to_html(index=False, border=1)}
<h2>Data Quality</h2>
{quality.to_html(border=1)}
<h2>Descriptive Statistics</h2>
{statistics.to_html(border=1)}
</body>
</html>
"""
with open(
f"{output_prefix}.html",
"w",
encoding="utf-8"
) as file:
file.write(html)
return overview, quality, statistics
# Example usage
df = pd.read_csv("sales.csv")
overview, quality, statistics = generate_eda_report(df)
print(overview)
print(quality)
print(statistics)
This function generates an HTML report and three CSV files. For production use, expand it to document the data source, analysis period, cleaning decisions, metric definitions, charts, and business interpretation. If column values are untrusted and the report will be published, escape user-provided text before inserting it into HTML.
19. Interpreting Findings and Writing Recommendations
An analytical report should explain what the results mean rather than merely listing numbers. It should distinguish observations from possible explanations and recommendations.
Example of an Analytical Finding
Observation: The regional summary indicates that Region A has the highest total sales among the regions included in the dataset.
Interpretation: Region A contributes the largest share of recorded revenue during the selected period. This may reflect a larger customer base, higher transaction values, or a different product mix.
Recommendation: Compare order volume, average order value, product mix, and profit margin across regions before deciding whether to increase marketing investment in Region A.
Limitation: The report covers only the available records and time period. It does not establish why one region performs better than another.
Questions an Analytical Report Should Answer
- What question does the analysis address?
- Which dataset and time period were used?
- What are the most important metrics?
- Which trends or differences are visible?
- Are findings affected by missing data or unusual observations?
- What limitations could influence the conclusions?
- What practical action is supported by the evidence?
20. Best Practices for EDA and Analytical Reporting
- Define the business question before beginning analysis.
- Preserve the original dataset before cleaning.
- Verify data types, units, time zones, and date ranges.
- Check missing values and duplicates before interpreting results.
- Use descriptive statistics and visualizations together.
- Choose chart types that match the analytical question.
- Do not interpret correlation as proof of causation.
- Define metrics and denominators clearly.
- Document exclusions, assumptions, and transformations.
- Use consistent number formats and readable chart labels.
- Protect confidential and personally identifiable information.
- Make the analysis reproducible with documented Python code.
- Verify exported files and report calculations before sharing.
21. Common Errors and Troubleshooting
| Problem |
Possible Cause |
Solution |
| FileNotFoundError |
Incorrect file path or unavailable file. |
Check the working directory and file location. |
| KeyError for a column |
The requested column does not exist. |
Inspect df.columns and verify spelling. |
| TypeError during calculations |
A numeric column contains text values. |
Inspect values and convert the column appropriately. |
| Unexpected totals |
Duplicates or incorrect aggregation rules. |
Check record uniqueness and the meaning of each row. |
| Charts omit observations |
Missing values or incompatible data types. |
Inspect missing values and confirm plotting behavior. |
| Correlation contains missing values |
Insufficient valid observations or constant columns. |
Check sample sizes, missing values, and variable variation. |
| Excel export fails |
An Excel-writing engine is unavailable. |
Install a compatible package, such as openpyxl. |
| HTML report is difficult to read |
The table is too wide or contains too many columns. |
Select relevant columns or divide the results into sections. |
22. Practical Exercises
Exercise 1: Dataset Inspection
Load a sales CSV file and display its first ten rows, dimensions, column names, data types, and descriptive statistics.
Exercise 2: Data Quality Report
Calculate missing-value counts and percentages, count duplicate rows, and list unique values in categorical columns.
Exercise 3: Univariate Analysis
Generate a histogram for sales amounts and a bar chart for product categories. Write two observations based on the results.
Exercise 4: Bivariate Analysis
Create a scatter plot of advertising spending and sales. Calculate their correlation and explain why the result does not prove causation.
Exercise 5: Regional Performance
Calculate total sales, average sales, and transaction counts by region. Identify the highest-performing region for the selected metric.
Exercise 6: Analytical Reporting
Export a dataset overview, data-quality summary, descriptive statistics, and at least one chart. Write a conclusion and two evidence-based recommendations.
23. Mini Project: Retail Sales EDA and Reporting
Project Objective: Analyze a retail sales dataset and produce a report covering business performance, data quality, product performance, and sales trends.
Suggested Dataset Columns
OrderID — unique order identifier.
OrderDate — transaction date.
Product — product name or identifier.
Category — product category.
Region — sales region.
Quantity — units sold.
Sales — recorded sales amount.
Profit — recorded profit, if available.
Project Steps
- Load the dataset and preserve an original copy.
- Inspect dimensions, column names, and data types.
- Check missing values, duplicates, and invalid dates.
- Validate quantity, sales, and profit values against business rules.
- Calculate descriptive statistics for numeric columns.
- Visualize sales distributions and monthly trends.
- Compare sales and profit by region and category.
- Calculate relevant KPIs using clearly defined formulas.
- Export summary tables and visualizations.
- Create an HTML or Excel report with findings, limitations, and recommendations.
Example: Regional and Category Summary
import pandas as pd
import matplotlib.pyplot as plt
df = pd.read_csv("retail_sales.csv")
df["OrderDate"] = pd.to_datetime(
df["OrderDate"],
errors="coerce"
)
regional_summary = df.groupby("Region").agg(
Total_Sales=("Sales", "sum"),
Average_Sales=("Sales", "mean"),
Order_Line_Count=("Sales", "count")
).sort_values("Total_Sales", ascending=False)
category_summary = df.groupby("Category").agg(
Total_Sales=("Sales", "sum"),
Total_Quantity=("Quantity", "sum")
).sort_values("Total_Sales", ascending=False)
print(regional_summary)
print(category_summary)
regional_summary.to_csv("regional_summary.csv")
category_summary.to_csv("category_summary.csv")
regional_summary["Total_Sales"].sort_values().plot(
kind="barh"
)
plt.title("Total Sales by Region")
plt.xlabel("Total Sales")
plt.ylabel("Region")
plt.tight_layout()
plt.savefig("sales_by_region.png", dpi=150)
plt.close()
The example assumes the specified columns exist in the CSV file. The Order_Line_Count counts non-missing sales values, not necessarily distinct orders. If one order contains multiple product lines, use the distinct OrderID count for order-level reporting.
24. Interview Questions and Answers
Q1. What is Exploratory Data Analysis?
Answer: EDA is the process of examining data using descriptive statistics, visualizations, and quality checks to understand its structure, distributions, patterns, relationships, and anomalies.
Q2. Which Python libraries are commonly used for EDA?
Answer: Pandas and NumPy support data manipulation and numerical calculations, while Matplotlib and Seaborn are commonly used for visualization.
Q3. What does pandas describe() do?
Answer: It generates descriptive statistics. Numeric columns typically include count, mean, standard deviation, minimum, quartiles, and maximum.
Q4. What is the difference between univariate and bivariate analysis?
Answer: Univariate analysis studies one variable at a time, while bivariate analysis examines two variables to explore their relationship or compare their behavior.
Q5. What is a correlation matrix?
Answer: It is a table of pairwise correlation coefficients for selected variables. It helps identify linear associations but does not establish causation.
Q6. When should a histogram be used?
Answer: A histogram examines the distribution of a numeric variable, including its spread, skewness, clusters, and potential outliers.
Q7. What is the purpose of groupby()?
Answer: It groups records according to one or more keys and supports calculations such as sums, means, counts, and medians for each group.
Q8. What is a KPI?
Answer: A Key Performance Indicator is a measurable value used to evaluate progress toward a defined business objective.
Q9. How can Python generate an analytical report?
Answer: Use pandas to calculate metrics and tables, Matplotlib or Seaborn to create charts, and export functions such as to_csv(), to_excel(), and to_html() to generate report files.
Q10. Why is data cleaning important before EDA?
Answer: Missing values, duplicates, incorrect types, and invalid entries can distort statistics and visualizations. Cleaning and documenting these issues improves the reliability of findings.
Q11. What is the difference between a finding and a recommendation?
Answer: A finding describes what the data shows. A recommendation suggests an action based on the finding, business context, and relevant limitations.
Q12. How can an EDA report be made reproducible?
Answer: Preserve the source data, document the steps, define metrics consistently, record assumptions, and use scripts or notebooks that can be rerun on the intended dataset.
25. Quick Revision Checklist
- Define the business question before starting analysis.
- Load data and inspect dimensions, columns, and data types.
- Review descriptive statistics and unique-value counts.
- Check missing values, duplicates, and invalid records.
- Use univariate analysis to examine individual variables.
- Use bivariate and multivariate analysis to explore relationships.
- Create appropriate histograms, bar charts, line charts, scatter plots, and heatmaps.
- Use grouping and aggregation to compare categories.
- Calculate KPIs with clear definitions and correct denominators.
- Investigate potential outliers instead of automatically deleting them.
- Export summary tables and charts in suitable formats.
- Document findings, assumptions, limitations, and recommendations.
- Verify the report before sharing it.
26. Conclusion
Exploratory Data Analysis is a fundamental step in the Python data-analysis workflow. It combines data inspection, descriptive statistics, data-quality checks, grouped calculations, and visualizations to help analysts understand what a dataset contains and which questions deserve further investigation.
Generating an analytical report extends this process by organizing results into meaningful metrics, charts, findings, and recommendations. A useful report explains the context, defines measurements, acknowledges limitations, and supports decisions with evidence rather than simply displaying numbers.
Additional Learning Resources